LESSONS LEARNED FROM 
FIVE YEARS OF BUILDING 
CAPTURE THE FLAG 


Vito Genovese 
DEF CON Beijing 2018 


HELLO! 


CAPTURE THE FLAG 


"(übe 
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1996 
first game 


2000 


formalized 
how it was run 


2002-2004 
ghettohackers 


2005-2008 
Kenshoto 


2009-2012 
ddtek 


2013-2017 


Legitimate 
Business 
Syndicate 


2018-? 


Order of the 
Overflow 


Two Distinct 
Formats 


6/16/2013 5:54:05 PM «global» PPP solved everything with two hours to spare! Call Heinz 
'cause y'all need to ketchup! 1 :49:00 left 
6/16/2013 5:51:03 PM >private< Your teammate PPP solved bob [OMGACM] for 4 points. e e 
6/16/2013 5:50:58 PM >private< Your teammate PPP solved bob [OMGACM] for 4 points. 
6/16/2013 2:11:12 PM «global» You should totally check out OMGACMS. 
16/2013 3:28:18 PM >private< Your teammate PPP solved tastycloud [gni 


Routards 
NOWHACKER-B105 


JEOPARDY STYLE 


Scoreboard 


JEOPARDY STYLE 


Prompt 


crackme1_f92e0ab22352440383d58be8f046bebe.quals.shallweplayaga.me:10001 


FILES 


“4a3b34dad0a707dbfc9283206720fa0f2b5a576c73557960b11506.tar.bz2 


JEOPARDY STYLE 


Solving 


@xC6C is a long subroutine, with many calls other subroutines; a closer look reveals 
that each of them gets as input a character of s, in order, and checks it is the 
"correct" one. 


Something helps even more: the exit code when no input byte is correct is 1, when the 
first one is correct is 2, and so on: 


0x93B: 
sub 93B proc near 
rdi, 9h 
short loc 94F 
rsp, 8 
nov edi, 1 
exit 
0x955: 


sub 955 prec near 
rdi, 65h 
shert loc 969 


sub 955 endp 


I did not investigate this any further, and wrote a python script to bruteforce the 
solution byte by byte. 


1 import string 
2 import subprocess 


JEOPARDY STYLE 


Points 


POTENT PUNABLES 


EM 70 Leo es Pequeno 
EM 94 badint 


E 69 peROPdo 

IM 62 mute 

M 142 insanity 

M 234 reeses revenge 


JEOPARDY STYLE 


DEF CON 
CTF Quals 


From 
SHA201 7 CTF 


Get file 


asby (100) - 331 solves 


Eindbazen team member asby has by far been putting the most energy and time in 
creating the SHA2017 CTF. To honor his dedication and all his effort we created this 


challenge as an ode to him. 


You can choose to reverse engineer this challenge or you can "asby" it. Good luck 
with the option you choose. 


ri asby.tgz 7422948a4034252d45cee02753b3d13b 


ASBY 


Determine 
goal 


2. xclient@5196d80f9b61: /asby (docker) 


xclient@5196d80f9Sb... 361 


xcliente5196d80f9b61:/asby$ wine asby.exe 
What is the flag? hach the planet 
Checking char 1:WRONG! 

What is the flag? Checking char 1:WRONG! 
What is the flag? Checking char 1:WRONG! 
What is the flag? ctfí 

Checking char 1:WRONG! 

What is the flag? flagi 

Checking char 1:CORRECT! 

Cheching char 2:CORRECT! 

Cheching char 3:CORRECT! 

Checking char 4:CORRECT! 

Checking char 5:CORRECT! 

What is the flag? flagix 

Cheching char 1:CORRECT! 

Checking char 2:CORRECT! 

Checking char 3:CORRECT ! 

Checking char 4:CORRECT! 

Checking char 5:CORRECT! 

Cheching char 6:WRONG! 

What is the flag? E 


Guess the 
correct input 


ASBY 


e Reverse engineer a Windows binary 
e Guess each character by hand 


e Write a program 


ASBY 


Write 
program 


& rb) asby.rb 


require ‘expect’ 


IO.popen("wine asby.exe", 
"r+") do |asby] 


start = "flag{024baa8ac03ef" 


asby.expect "What is the flag? " 
asby.puts start + "\r" 
start.length.times do |n| 
got = asby.expect(/Checking char \d+:CORRECT!\r\n/, 1) 


p got 
end 


actual flag = '' 
flag chars = 34(0123456783abcdef \} } 


loop do 
asby.flush 
flag chars.each do |candidate| 
print "\r#{candidate}" 
asby.expect "What is the flag? " 
asby.puts(start + actual flag + candidate + "\r”) 
pruwr'.* 
(start. Length + actual flag.length).times do |n] 
asby.expect "Checking char #{n+1}:CORRECT!\r\n" 
end 


if asby.expect(/Cheching char \d+:\w+\!\r\n/, 1).join =~ /CORRECT/ 
puts "Ar" 
actual flag += candidate 
puts start + actual flag 
if “14º == candidate 
puts start + actual flag 
exit 
end 
break 
end 
end 
end 


end 


-:--- asby.rb ALL (44,0) Git:master (Ruby RuboCop YARD company WS RE 


require ‘expect’ 


I0.popen( wine asby.exe”, 
"r+") do lasbyl 


start = "flagí8024baa8ac03ef" 


asby.expect "What is the flag? " 
asby.puts start + "Ar" 
start.length.times do Inl 
got = asby.expect(/Checking char \d+:CORRECT!\r\n/, 1) 


p got 
end 


flag chars = 2w10 123456789abcdef\}} 


loop do 
asby.flush 
flag chars.each do |candidate| 
print "\r#{candidate}" 
asby.expect "What is the flag? " 
asby.puts(start + actual flag + candidate + "\r") 
printh”.* 
(start. Length + actual flag.length).times do Inl 
asby.expect "Checking char #{n+1}:CORRECT!\r\n" 
end 


if asby.expect(/Cheching char \d+:\w+\!\r\n/, 1). join =~ /CORRECT/ 
puts H NE LI] 
actual flag += candidate 
mite start + actiial flag 


end | 
actual flag = ‘’ 
flag chars = %uf0 123456 789abcdef\}} 


loop do 
asby.flush 
flag_chars.each do |candidate| 
print "\r#{candidate}" 
asby.expect "What is the flag? " 
asby.puts(start + actual flag + candidate + "\r") 
printh”.* 
(start. length + actual_flag.length).times do In| 
asby.expect “Checking char #{n+1}:CORRECT!\r\n" 
end 


if asby.expect(/Checking char \d+:\w+\!\r\n/, 1). join =~ /CORRECT/ 
puts An 
actual flag += candidate 
puts start + actual flag 
if “4” == candidate 
puts start + actual flag 
exit 
end 
break # out of the candidate Loop 
end 


ASBY 


Get solution 


3. xclientoa84249bfa09c: /asby (docker) 


xclient@aB4249bfa0... 8$1 


flag{024baa8ac03ef22fdde61c0f11 
"o HM 
EEE LSU 
TlagfezbootachSefz2 des 100111068 


flag{024baa8ac03ef22fdde61c0f11069f 
ER 
flagi024baa8ac03ef22fdde61c0f11069f2 
[^ 
flag{024baa8ac03ef22fdde61c0f11069f2f 


= 
flagi024haa8ac03ef22fdde61cof11069f2f; 
dasby.rb:24:in ‘write’: Broken pipe (Errno::EPIPE) 
from asby.rb:24:in ‘puts’ 
from asby.rb:24:in ‘block (3 Levels) in <main>’ 
from asby.rb:21:in ‘each’ 
from asby.rb:21:in ‘block (2 levels) in <main>” 
from asby.rb:19:in ‘Loop’ 
from asby.rb:19:in ‘block in <main>* 
from asby.rb:3:in *popen” 
from asby.rb:3:in “<main>’ 
xclLient@384249bfa09c: /asby$ | 


Get points 


JEOPARDY STYLE 


1. Get challenge 
2. Solve it 
3. Get points 


7 p^ 
Level nr 
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| fi | 
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ATTACIC- DEFENSE 


eReverse engineer 
ePatch flaws 
eExploit others 
eDont break it 


ATTACIC- DEFENSE 


Ls 


ATTACIC- DEFENSE 


deposit 


ATTACIC- DEFENSE 


ATTACIC- DEFENSE 


ATTACIC- DEFENSE 


Failed — SCorebor > 


ATTACIC- DEFENSE 


DEF CON 
CTF Finals 


a 3. xclient@a84249bfa089c: /asby (zsh) 
xclient@a84249bfa0... 381 


This service implements a rubix cube. Solve the cube and win. 
Give me 54 Bytes(ie: 53,12,5,etc): «input 54 comma-delimited chars» 
Top 
R6 R7 BO 


L1 T4 R5 
AB B7 R2 


Left 


ES EY LE 
TS L4 B1 
B& B3 F8 


Front 


TO A7 FO 
F7 F4 M 
A6 R3 L2 


Bottom 
AZ F5 Fb 
B5 B4 A3 
T8 A5 F2 


Right 


R8 F3 L6 
T1 R4 T? 
T2 L3 B2 


Bach 


B6 T3 T6 
F1 A4 R1 
RO L5 AS 


Action(U,U’,D,D’,L,L’,R,R’,F,F’,B,B’): 


RUBIX 


54 Rubik's cube instructions 


...becomes shellcode 


LAB RATS ON RUBIX 


Lab RATs posted a write-up: 


https://blog.rpis.ec/2017/08/defcon- 
Finals-2017-introduction-rubix.html 


LAB RATS ON RUBIX 


1. write 9-bit to 8-bit netcat 
2. analyze 9-bit strings in libc 
3. symbolize libc 


4. figure out how main() 
gets called 


LAB RATS ON RUBIX 


Now the 
actual analysis 
starts... 


ATTACIC- DEFENSE 


eHow is it supposed to 
work? 


eHow can we attack it? 


eHow can we defend it? 


ATTACIC- DEFENSE 


eGet points by capturing flags 


e| ose points by having flags 
captured 


e| ose lots of points by failing 
checks 


Complicated, 
Frustrating, 
fun! 


Extremely 
ambitious 


e Running Smoothly 
eFair Contest 


eFun Challenges 


RUNNING SMOOTHLY 


RUNNING SMOOTHLY 


Starts early 


RUNNING SMOOTHLY 


Who's on the 
team? 


LEGITIMATE BUSINESS SYNDICATE 


eHalf 2005-2007 
university team 


eHalf 2012 coworkers 


LEGITIMATE BUSINESS SYNDICATE 


e August 2012: ddtek steps down 


eDecember 2012: Gyno starts 
recruiting 


eFebruary 2013: Proposal 
submitted 


eMarch 2013: Proposal accepted 


LEGITIMATE BUSINESS SYNDICATE 


e"Reverse engineers" 3/4 of the group 
eDifferent specialties 
e Radio: 2014, badger 


eHardware: 2015, the year of single- 
board computers 


eEsoteric computing: 2017, cLEMENCy 


LEGITIMATE BUSINESS SYNDICATE 


100% dependent 
on Selir s amazing 
infrastructure 


LEGITIMATE BUSINESS SYNDICATE 


| started for the 
database backed | 
web application 


TEAM BUILDING 


People grow 
and change 


TEAM BUILDING 


Roles grow 
and change 


TEAM BUILDING 


e\Vho do you know? 
e\Vho do you trust? 
e\Vho do you like? 


COMMUNICATION 


“It's good." 


COMMUNICATION 


async (chat) is great 


weekly meetings are great 


SMOOTH OPERATION 


Support your 
team 


SMOOTH OPERATION 


CTF software 
is software 


SMOOTH OPERATION 


Automate 
testing and 
deployment 


e Running Smoothly 
eFair Contest 


eFun Challenges 


FAIR CONTEST 


FAIR CONTEST 


CTF IS 


computer 
hacking 


FAIR CONTEST 


CTF IS 
computer 
system 


FAIR CONTEST 


Hack the right 
thing the ~ 
Wrong way 


FAIR CONTEST 


Hack the 
wrong thing 


Fix a thing 
the wrong. 
Way 


FAIR CONTEST 


Restrict 
players more 


QUALIFIERS 


eServices on separate hosts 


eMultiple hosts in different locations 
eConnections get separate container 
exinetd and runc 
e| imit system calls 


eseccomp 


FINALS 


More complex game 


More complex problems 


FINALS 


eKeep the game about 
reverse engineering 


e(Not OS administration) 


e2013: unprivileged team 
account, unprivileged 
service accounts 


e2014: understood 
"Superman defense" better 


SUPERMAN DEFENSE 


eBlock opponent IPs 


ePrevent reading the flag 


CYBER GRAND CHALLENGE 


US Defense Advanced 
Research Projects 
Agency (DARPA) 


project starting in 
2014 


CYBER GRAND CHALLENGE 


CTF for 
autonomous 
computers 


CYBER GRAND CHALLENGE 


Extremely 
formalized 


CHALLENGE BINARIES 


e CBs" 

e32-bit i386 

eSpecial CGCEF executable format 
e| imited system calls 


eNo retained state 


PROOF or VULNERABILITY 


e PoVs" 
e32-bit i386 CGCEF 


eDemonstrate a vulnerability: 


eRegister control 
eMemory disclosure 


eRun by scoring system 


OFFLINE EVALVATION 


eTeam interface gives out binaries 


eTeam interface collects replacement 
CBs, PoVs 


eRuns availability checks and PoVs in 
isolation 


eDesigned for reproducibility and 
audibility 


e2015: restrict system calls 


e2016: use CGC game 
format 


e2017: everything in 
limited emulator 


Release 
scoring 
information 


Think about 
accessibility 


e Running Smoothly 
eFair Contest 


eFun Challenges 


FUN CHALLENGES 


Break 
expectations 


DOSFUNTU 


e Discover that it's a DOS binary 
eDebug and patch IDA Pro 


eStart actual reverse engineering 


BADGER 


eMSP-430 on physical 
hardware 


ecustom CDMA radio 
network 


Legitimate 
Business 
di 


emas * 


OTT 


mii 5 

co m) re - 

ok 0 : 
2 


Cohen eh te erent 


= uuu.legitbs.net 
> PCB Made in 


ONSENSUS EVALVATIO 


eCGC's big attack-defense 
Innovation 


eEveryone sees everyone else's 
patched binaries 


eExplosion in number of 
binaries that need reversing 


1666 curs / CRACKMEZOOO 


Push teams into 
automated analysis 


Hundreds of binaries 


CONSENSUS EVALUATION IN 2016 


Player asks about losing points 


Service being attacked, that's 
why 


"But we're using the same 
binariess as the winning team" 


CONSENSUS EVALUATION IN 2010? 


Rubix expected shellcode to work in 
availability checks 


Defenders would add checks to block 
“evil” or allow "good" shellcode 


Attackers would build new shellcode to 
pass checks 


"Felt like a multiplayer game against 
humans" 


e Running Smoothly 
eFair Contest 


eFun Challenges 


Still more to 
learn! 


More work 
ahead of us 


Opportunity 
to grow for 
more players 


Best way Lo 
learn is to do 


Five years with 

the best group 

of people I've 
ever worked with 


Five years 
building a contest 
for the friendliest 
and smartest 
community | know 


Thanks for 
making it 
amazind! 


(1) www.oooverflow.io 


Welcome, young hacker. 


Who are we? 


We are the Order of the Overflow. You can read about our 
philosophy here or reach us at team@oooverflow.1o. 


When are the quals? 


The qualifying event for DEF CON CTF 2018 Finals will start on 
00:00 UTC on May 12th and go for 48 hours. 


What are the pre-qualifying events? 


The winners of the following events will be automatically pre- 
qualified for finals: 


DEF CON CTF - 28 July 2017 - prequalified: PPP 


vito@legitbs.net 
@vito lbs 


